Method and apparatus for face detection/recognition systems

ABSTRACT

A computer implemented method for detecting an attempt to spoof and facial recognitions apparatus determines for a plurality of spatially separated regions of a surface, a respective measure of at least one vital sign. A determination is made from the respective measures of at least one vital sign, homogeneity information associated with the respective measures, the homogeneity information is used to determine if said spatially separate regions of said surface are living tissue.

CROSS-REFERENCE TO PRIOR APPLICATIONS

This application is the U.S. National Phase application under 35 U.S.C.§ 371 of International Application No. PCT/EP2017/066219, filed on Jun.29, 2017, which claims the benefit of European Application Serial No.16305819.1, filed Jun. 30, 2016. These applications are herebyincorporated by reference herein, for all purposes.

FIELD

Some embodiments relate to a method and apparatus for use in particularbut not exclusively in face detection/recognition systems.

BACKGROUND

In an effort to increase security, facial recognition is being used moreand more in various types of applications. The expected added value offacial recognition is the increased prevention of counterfeiting.

For example, cameras are frequently used in security systems such assurveillance equipment and more and more they are used to photograph orfilm people for identification and logging purposes as they pass throughcheck-points such as passport controls or entries to secure areas.

Current face-recognition systems have difficulty in detecting where asubject is a photograph of another person or where a subject is wearinga mask. The act of a subject generating false positive from a facerecognition system in this manner is known as “spoofing”.

Spoofing is problem enough for situations where there is humanintervention and the camera is just for logging purposes because themask will often be detected by the security personnel and the only errorwill be in the log files. Showing a printed image will not work in thissituation. That said, sometimes people wearing masks could slip throughcompletely undetected. In a situation where access were to be grantedautomatically by a face recognition system, or where the security personis on the other side of the camera, this is a problem as no-one is thereto see the person in the flesh and so artificial representations of ahuman face can work. For example, a wearable active display like acomputer tablet may be used.

Therefore it is desirable to reduce the reliance on the presence ofsecurity personnel and provide a way of detecting the spoofing attemptusing the camera system itself.

There may also be less critical situations where it remains desirable toreduce the opportunity for fooling a camera system by the use of a mask.

SUMMARY

According to one aspect, there is provided a computer implemented methodcomprising: determining for a plurality of spatially separated regionsof a surface, a respective measure of at least one vital sign;determining from said respective measures of at least one vital sign,homogeneity information associated with said respective measures; andusing said homogeneity information to determine if said spatiallyseparate regions of said surface are living tissue.

The method may comprise identifying a face area in a sequence of videoframes, at least one of said spatially separated regions beingpositioned in said face area, said surface comprising at least partiallysaid face area. This may be advantageous as it allows the system tocheck across multiple regions of a face, ensuring that a spoofer is notwearing a partial mask.

The respective measure of at least one vital sign may comprise aheart-beat signal, the method may further comprise extracting, from thesequence of video frames, a heart-beat signal for each of the spatiallyseparated regions so as to obtain a plurality of heart-beat signals.This may be advantageous as it allows the system to check whether theface is made of living tissue.

The method may comprise determining the homogeneity information bycomparing the heart-beat signals. This may be advantageous as it allowsthe system to compare the measured values across a face, minimising theability of a spoofer to trick the system by simulating a heart-beat.

The use of the homogeneity information, may comprise comparing thehomogeneity information against a limit. The limit may be a pre-setvalue. This may be advantageous as it allows the system to check thevalues measured against those previously given to the system. This maygive an operator an ability to tune the system to be more or lesssensitive to a potential spoofers.

The homogeneity information may be determined by extracting a heart-ratefrom the heart-beat signals and combining the heart-rates from aplurality of the spatially separate regions and determining at least oneof a standard deviation and a maximum value of the combined heart-rates.The maximum value may be the maximum value of a histogram distribution.This may be advantageous as it may allow the system to check forhomogeneity across different areas of the face, which may further theability of the system to detect a partial mask.

The determining of the homogeneity information, may comprise finding acorrelation between at least one pair of the heart-beat signals.

The determining of the homogeneity information may comprise extractingheart-rates for a spatially separate region from a plurality of timesegments and determining a standard deviation of the heart-ratesextracted from each time segment.

The method may comprise determining a colour vector for a spatiallyseparate region for each of a plurality of time segments and determiningvariations of the colour vectors over the plurality of time segments.This may be advantageous as it allows the system to, for example,calculate the HR-related colour variation (HR-axis) to increase thesystem's ability to detect a spoofed face.

The colour vector for the spatially separate region for each of aplurality of time segments may be obtained from a haemoglobin absorptionspectrum.

The method may comprise determining an area in the sequence of videoframes where respiration is measurable, extracting a respiration signaland extracting a correlation between said respiration signal andheart-beat signal and comparing the correlation to a limit. This may beadvantageous, as it allows the system to use a differentphotoplethysmographic (PPG) waveform which a spoofed face may notpossess.

According to another aspect there is provided a method of preparing aspoofing detection unit comprising performing a teaching procedure, theteaching procedure comprising performing any of the methods above on avideo sequence containing images of a real face using first homogeneityinformation, performing any of the above method on a video sequencecontaining images of a spoof face using second homogeneity information,and setting a limit to a value lying between the first and secondhomogeneity information.

According to another aspect there is provided a spoofing detectionapparatus comprising at least one processor configured to: determine fora plurality of spatially separated regions of a surface, a respectivemeasure of at least one vital sign; determine from said respectivemeasures of said at least one vital sign, homogeneity informationassociated with said respective measures; and use said homogeneityinformation to determine if said spatially separate regions of saidsurface are living tissue.

The apparatus may comprise an input to receive a sequence of videoframes and the at least one processor may be configured to identify aface area in a sequence of video frames, to identify a set of spatiallyseparate skin portions in the face area, and to extract a heart-beatsignal for each of the spatially separate skin portions and from eachheart-beat signal, to extract a heart-rate.

The at least one processor may be configured to calculate a valid vitalsign metric and compare the valid vital sign metric against a limit todecide if the sequence of video frames containing the face area has beencaptured directly from the face of a living person.

The at least one processor may be configured to identify a face area ina sequence of video frames, at least one of said spatially separatedregions being positioned in said face area, said surface comprising atleast partially said face area.

The respective measure of at least one vital sign may comprise aheart-beat signal, the at least one processor may be configured toextract, from the sequence of video frames, a heart-beat signal for eachof the spatially separated regions so as to obtain a plurality ofheart-beat signals.

The at least one processor may be configured to determine thehomogeneity information by comparing the heart-beat signals.

The at least one processor may be configured to compare the homogeneityinformation against a limit.

The at least one processor may be configured to determine thehomogeneity information by extracting a heart-rate from the heart-beatsignals, combining the heart-rates from a plurality of the spatiallyseparate regions, and determining at least one of a standard deviationand a maximum value of the combined heart-rates. The maximum value maybe the maximum value of a histogram distribution.

The at least one processor may be configured to find a correlationbetween at least one pair of the heart-beat signals.

The at least one processor may be configured to extract heart-rates fora spatially separate region from a plurality of time segments anddetermine a standard deviation of the heart-rates extracted from eachtime segment.

The at least one processor may be configured to determine a colourvector for a spatially separate region for each of a plurality of timesegments and to determine variations of the colour vectors over theplurality of time segments.

The colour vector for the spatially separate region for each of aplurality of time segments may be obtained from a haemoglobin absorptionspectrum.

The at least one processor may be configured to determine an area in thesequence of video frames where respiration is measurable, extract arespiration signal, extract a correlation between said respirationsignal and heart-beat signal, and compare the correlation to a limit.

According to another aspect there is provided a visual recognitionsystem which may comprise: a video camera operable to capture a sequenceof video frames; a spoofing detection apparatus as previously described,and an alert unit configured to generate an alert if a spoofing attemptis detected.

According to another aspect, there is provided a method for detecting anattempt to fool a visual recognition system comprising: identifying aface area in a sequence of video frames; identifying a plurality ofspatially separate skin portions, at least one skin portion beingpositioned in the face area; extracting, from the sequence of videoframes, a heart-beat signal for each of the spatially separate skinportions so as to obtain a plurality of heart-beat signals, eachheart-beat signal being a member; calculating a valid vital sign metricfrom a heart beat signal by performing a comparison between members ofthe plurality; comparing the valid vital sign metric against a limit todecide if the video sequence containing the face area has been captureddirectly from a living person.

According to some aspects, there is provided a program productcomprising a computer-readable storage device including acomputer-readable program for providing a computer-implemented game,wherein the computer-readable program when executed on a computer causesthe computer to perform any one or more of the method steps describedpreviously.

A computer program comprising program code means adapted to perform themethod(s) may also be provided. The computer program may be storedand/or otherwise embodied by means of a carrier medium.

In the above, many different embodiments have been described. It shouldbe appreciated that further embodiments may be provided by thecombination of any two or more of the embodiments described above.

Various other aspects and further embodiments are also described in thefollowing detailed description and in the attached claims.

BRIEF DESCRIPTION OF THE DRAWINGS

The above, as well as additional objects, features and advantages of thedisclosed devices, systems and methods, will be better understoodthrough the following illustrative and non-limiting detailed descriptionof embodiments of devices and methods, with reference to the appendeddrawings, in which:

FIG. 1 shows a system of some embodiments;

FIG. 2 shows schematically shows functional blocks of the signalprocessor of FIG. 1;

FIG. 3 shows a method of some embodiments;

FIG. 4 shows a face with selected regions of interest;

FIG. 5 shows selected faces in an image and a corresponding heart ratemap;

FIG. 6 shows in more detail the respective heart rate maps andassociated histograms; and

FIG. 7 shows a method of some embodiments.

DETAILED DESCRIPTION OF EMBODIMENTS

Some embodiments provide a computer implemented method comprising:determining for a plurality of spatially separated regions of a surface,a respective measure of at least one vital sign; determining from saidrespective measures of at least one vital sign, homogeneity informationassociated with said respective measures; and using said homogeneityinformation to determine if said spatially separate regions of saidsurface are living tissue. This may be advantageous as it may remove theneed for a person to be present during the digital identificationprocess, by utilising a detected vital sign of a person attempting topass the facial recognition system. Embodiments may also allow thesystem to check whether a person is wearing a partial mask, wherein thewearer has some of their face exposed.

In the following description, the same references designate likeelements.

Some embodiments relate to a method and apparatus for an improvedbiometric identifier for face detection/recognition systems by utilizinghuman vital sign signals from a plurality of areas of a face.

Some embodiments utilize the presence of photoplethysmographic (PPG)waveforms caused by heart pulsation to determine if the face of asubject analysed by a facial recognition/detection system is that of areal face. Photoplethysmography (PPG) is an optical method involving theextraction of a signal indicative of a physiological process or vitalsign from a subject. The vital signs may be obtained by the facedetection system though a sequence of images. This allows the facedetection system to ascertain whether the face being imaged is real or aspoof (a photograph or a mask).

In some embodiments a simple approach may be to qualify a face as realif a photoplethysmographic waveform can be detected. In some scenariosthis method may not be sufficient because a subject may use parts of aface picture (e.g. the eyes), or part of a mask in conjunction withparts of their own face to spoof the recognition/detection system.

Some embodiments may use an approach involving spatially resolvedphotoplethysmographic waveforms in combinations with a homogeneitymeasure to qualify a detected face as real or not. As the samephotoplethysmographic pulse waveform is homogeneously distributed acrossthe face being measured, utilizing spatially resolvedphotoplethysmographic waveforms may aid in the correct identification ofa face as being real.

In some embodiments a check for consistency in the pulse waveforms overtime may be determined. Such checks may further reduce the possibilityof counterfeiting the detection/recognition system.

The method may comprise identifying a face area in a sequence of videoframes, at least one of said spatially separated regions beingpositioned in said face area, said surface comprising at least partiallysaid face area. This may be advantageous as it allows the system tocheck across multiple regions of a face, ensuring that a spoofer is notwearing a partial mask.

The respective measure of at least one vital sign may comprise aheart-beat signal, the method may further comprise extracting, from thesequence of video frames, a heart-beat signal for each of the spatiallyseparated regions so as to obtain a plurality of heart-beat signals.This may be advantageous as it allows the system to check whether theface is made of living tissue.

Some embodiments identify the face region of a subject in a series offrames through the capture of a video stream. Some embodiments may thenidentify separate sub-regions in the face of a subject from whichmultiple photoplethysmography (PPG) streams may be extracted. Thehomogeneity of these multiple waveforms may then be calculated. Forexample, heart-rates (HRs) may be extracted and checked for consistencyover time. The results of these checks may then be utilized to determineif the face of a subject is real.

In some embodiments, the method for an improved biometric identifier forface detection/recognition system may comprise:

-   -   identifying a face area in a sequence of video frames;    -   identifying a set of spatially separate portions in the face        area;    -   extracting a heart-beat signal for each of the portions;    -   comparing the extracted signals and calculating a homogeneity        metric; and    -   comparing the similarity metric against a limit to decide if the        face is real.

The method may comprise determining the homogeneity information bycomparing the heart-beat signals. This may be advantageous as it allowsthe system to compare the measured values across a face, minimising theability of a spoofer to trick the system by simulating a heart-beat.

The use of the homogeneity information, may comprise comparing thehomogeneity information against a limit. The limit may be a pre-setvalue. This may be advantageous as it allows the system to check thevalues measured against those previously given to the system. This maygive an operator an ability to tune the system to be more or lesssensitive to a potential spoofers.

The homogeneity information may be determined by extracting a heart-ratefrom the heart-beat signals and combining the heart-rates from aplurality of the spatially separate regions and determining at least oneof a standard deviation and a maximum value of the combined heart-rates.The maximum value may be the maximum value of a histogram distribution.This may be advantageous as it may allow the system to check forhomogeneity across different areas of the face, which may further theability of the system to detect a partial mask.

The determining of the homogeneity information, may comprise finding acorrelation between at least one pair of the heart-beat signals.

The determining of the homogeneity information may comprise extractingheart-rates for a spatially separate region from a plurality of timesegments and determining a standard deviation of the heart-ratesextracted from each time segment.

In some embodiments, options to check the homogeneity of thephotoplethysmography (PPG) waveforms may comprise one or more of:

-   -   the standard deviation of heat-rates;    -   the maximum of a histogram of the heart-rates;    -   the correlation of heart-rates;    -   the standard deviation of variations in heart-rate over time        segments; and    -   the average distance of heart-rate related colour vector        variations over time.

The method may comprise determining a colour vector for a spatiallyseparate region for each of a plurality of time segments and determiningvariations of the colour vectors over the plurality of time segments.This may be advantageous as it allows the system to, for example,calculate the HR-related colour variation (HR-axis) to increase thesystem's ability to detect a spoofed face.

The colour vector for the spatially separate region for each of aplurality of time segments may be obtained from a haemoglobin absorptionspectrum.

In some embodiments the average distance of heart-rate related colourvector may be Euclidean. In other embodiments the temporal variation inan observed light spectrum may not be consistent over time and/or it maydiffer considerably from the expected heart-rate-induced variation oflight spectrum, based on the absorption spectrum of haemoglobin.

The homogeneity of the photoplethysmography (PPG) waveforms may bemeasured using one or more of these options in any combination. In someembodiments all of the above options are used. In other embodiments theoptions involving temporal segmenting are preferentially used incombination.

In some embodiments the relative amplitudes, and shapes of thephotoplethysmography (PPG) waveforms may be used.

In some embodiments controlled lighting may be utilized to illuminatethe face of a subject in a uniform diffuse manner.

In some embodiments specular reflections may be reduced through the useof a source that is polarized, for example having a cross-polarizer onthe camera lens. It should be appreciated that other types ofpolarization may be used.

In some embodiments the lighting may comprise infrared (IR)electromagnetic waves. Using infrared light may improve sensitivity andmake the presence of the system and the running of a scan less obviousto a subject. It should be appreciated that other wavelengths ofelectromagnetic waves may be used in the alternative or additionally.

Some application may favour a particular type of radiation. For exampleautomotive applications may favour IR radiation.

Certain physiological processes can be observed via skin reflectancevariations. The human skin can be modelled as an object with at leasttwo layers, one of those being the epidermis (a thin surface layer) andthe other the dermis (a thicker layer underneath the epidermis). Acertain percentage 5% of an incoming ray of light is reflected at theskin surface. The remaining light is scattered and absorbed within thetwo skin layers in a phenomenon known as body reflectance (described inthe Dichromatic Reflection Model). The melanin, typically present at theboundary of epidermis and dermis, behaves like an optical filter, mainlyabsorbing light. In the dermis, light is both scattered and absorbed.The absorption is dependent on the blood composition, so that theabsorption is sensitive to blood flow variations. The dermis contains adense network of blood vessels, about 10% of an adult's total vesselnetwork. These vessels contract and expand according of the blood flowin the body. They consequently change the structures of the dermis,which influences the reflectance of the skin layers.

It is possible to detect and extract signals which have some periodiccontent in these changes and from that obtain a result such as afrequency in the case of periodic processes. For example, a subject maybe illuminated with light and filmed using a video camera. By analysingchanges in the values of corresponding pixels between frames of thesequence of images, a time-variant signal can be extracted. This signalmay be transformed into frequency-like domain using something like aFast Fourier Transform and from the frequency-domain spectra, a valuefor the subject's heart-rate and/or respiration rate (rate of breathing)may be arrived at as a physiological measurement. These physiologicalmeasurements are often called vital signs. Any one or more vital signsmay be used with embodiments. Any vital sign may be used in someembodiments.

The method may comprise determining an area in the sequence of videoframes where respiration is measurable, extracting a respiration signaland extracting a correlation between said respiration signal andheart-beat signal and comparing the correlation to a limit. This may beadvantageous, as it allows the system to use a differentphotoplethysmographic (PPG) waveform which a spoofed face may notpossess.

In some embodiments a gating check on the illumination may be utilized,which may include an active illumination system where frames areconsecutively acquired with and without active illumination. The gatingcheck may be performed to remove the impact of ambient lighting. Beforebeing analysed by the algorithm, the consecutive images (activelight+ambient vs ambient light only) may be subtracted to use an imagecontaining only the active light to improve the reliability of thesystem.

In some embodiments the relative phases of the photoplethysmography(PPG) waveforms may be checked. Checking the phases of thephotoplethysmography (PPG) waveforms may improve the reliability of thesystem as the waveforms should be close in phase and well correlated.

Reference is now made to FIG. 1 which shows an embodiment of a facedetection system 100 and a subject 101. The face detection system 100comprises an electromagnetic radiation source 102. It should beappreciated that the electromagnetic radiation source 102 may emit anysuitable wavelength of light or infrared, as previously mentioned. Insome embodiments, the electromagnetic radiation 102 may be omitted andambient light may be used. An electromagnetic radiation detector 103 isprovided. It should be appreciated that the light electromagneticradiation 103 may detect any suitable wavelength of light, dependent onthe electromagnetic radiation source/ambient light.

In some embodiments, the electromagnetic radiation detector may be acamera. In some embodiments, the camera may be a video camera. It shouldbe appreciated, that in other embodiments, any other suitable detectormay be used.

The system comprises a signal processor 104. The signal processor maycomprise at least one processor 105, at least one memory 106, and aninterface 107. The interface is configured to receive the input imagesand to provide an output. Some of the types of output which may beprovided by some embodiments will be described later.

Reference is now made to FIG. 4 which schematically shows a face 13which has be captured over a series of images. There are a number ofregions of interest 11 on the face. For each of these regions, the heartrate 12 is determined.

Reference is now made to FIG. 2 which schematically shows functionalblocks of the signal processor. This will be described in conjunctionwith FIG. 3 which shows the method of some embodiments.

In step S201 the camera captures a set of consecutive frames which arestored in memory for processing. Function block 21 causes the set offrames to be stored in memory. In some embodiments the set ofconsecutive frames may be temporally spread over a given period. Thegiven period may be any suitable length of time. For example the periodmay be a 2 second period. In some embodiments the frames may be sampledat any suitable frequency, for example 7 Hz or a similar frequency. Insome embodiments a time period of 2 seconds may be used to cover thefrequency range of a human heart-rate (for example, 0.5 Hz-3.5 Hz). Insome embodiments a 7 Hz sampling rate may be used to respect the Nyquistsampling theorem.

In step S202 on a per frame basis, a face detection or recognitionalgorithm is applied. This is performed by the face detection functionblock 22. In some embodiments, this step is performed to give a regionof interest containing a face candidate on which the following part ofthe processing is operated. This is the face 13 shown in FIG. 4.

In step S203 spatially resolved photo-plethysmography (PPG) signals withrespect to the detected face are obtained. This is performed by thephoto-plethysmography function block 23. In some embodiments to obtainspatially resolved photo-plethysmographic waveforms, the face region maybe divided into small sub-parts (for example, squares of 10×10 pixels)and processed to extract a waveform from each sub-part. These are thesmaller regions of interest shown in FIG. 4. The photo-plethysmographicwaveforms are referenced 12 in FIG. 4.

In step S204 a homogeneity measure for the PPG signals is obtained. Thisis performed by the homogeneity block 24. In some embodiments, once thewaveforms are extracted a homogeneity measure may be computed. In someembodiments the homogeneity measure allows the waveforms to be verifiedas having the same characteristics.

The homogeneity measure may be derived in one or more of the followingways:

a. In some embodiments from each waveform the heart rate is extractedusing Fourier Transform. Then the homogeneity measure H is calculatedas:H1=standard deviation(all_HR_extracted)  (Eq.1)

b. In some embodiments from each waveform the heart rate is extractedusing Fourier Transform. A histogram of the HR extracted is build andnormalized. The homogeneity measure is then defined as:H2=max(Histo(all_HR_extracted))  (Eq.2)

c. In some embodiments all the waveforms are correlated (one with allothers) to build a correlation matrix C. The homogeneity measure is thendefined as:H3=mean(C_Significant)  (Eq.3)

where C_Significant represents all correlation C with a p-value below apredetermined threshold.

d. In some embodiments waveforms are subdivided in multiple (possiblyoverlapping) time intervals, where for each time interval the HR isextracted. For example, to calculate the variation of the HR over time(temporal consistency) the standard deviation of the HR values can becalculated:H4=standard deviation(consecutive_HR_extracted)  (Eq.4)

e. In some embodiments for the HR values of the time intervals that werederived under d), the direction (vector) in 3D RGB (red/green/blue)colour space is calculated of the HR-related colour variation (HR-axis).For example, the variation of this direction (vector) over time(temporal consistency) can be calculated. For example, a method fordoing this may be to first calculate the mean of the directions(vectors). The average (Euclidean) distance of the individual vectors tothis mean vector may then expresses the variation. The difference of thevariation in observed light spectrum to the expected heart-rate inducedvariation in the light spectrum (reference HR vector in colour spacebased on absorption spectrum of haemoglobin) may alternatively oradditionally be used.H5=mean(dist(consecutive_HR_RGB_vectors-mean(consecutive_HR_RGB_vectors)))  (Eq.5)

In step S205 it is determined whether the measured PPG signals aregreater than those of threshold values. This is performed by thethreshold block 25. In some embodiments there is checking if thehomogeneity measure is above or below a pre-determined threshold valueT. For example, if the measured value is found to be above the thresholdvalue, the face contained in the region of interest is qualified as“real-face”.

In step S206, it is determined whether the detected signals meet thethreshold criteria and a decision is made. This is performed by thethreshold block 25.

In step S207, it is determined that the detected face is a spoof. Thisis performed by the threshold block 25.

In step S208, it is determined that the detected face is a real face.This is performed by the threshold block 25.

A suitable output may be provided depending on the determination made instep S206. For example a visual and/or audible alarm may be provided. Insome embodiments, the output may be a control output which may be toopen a gate or door if it determined that the detected face is a realface.

In some embodiments, a facial recognition algorithm may also beperformed. The facial recognition algorithm may only be passed if it isdetermined that the detected face is a real face.

In some embodiments, heart rate related properties may be calculated andused as feature values. This may be in addition or as an alternative tothe homogeneity value. For example: the (relative) amplitude of thepulse variation; the temporal shape of the pulses; and/or the presencein the frequency spectrum of higher harmonics of the fundamental pulsefrequency may be used.

In some embodiments to reduce the effect that the variation inenvironmental light has in distorting the extracted heart rate (HR)signal, a combination of light sources may be added illuminating thehuman face. For example, these light sources may generate a diffuse,uniformly spatially distributed, and evenly distributed over the rangeof spectral sensitivity of the camera.

In some embodiments to improve the spatial homogeneity of the heart rate(HR) over the face, the amount of specular reflection from the skin maybe reduced. This may be achieved for example where the detected light iscross-polarized. A polarizer may be placed in-front of the light source,and another polarizer may be placed in-front of the camera, thepolarization direction of both polarizers may be chosen such that theyare orthogonal with respect to each other.

In some embodiments infrared light may be used for illumination of thesubject. The camera sensitivity may be in the infrared spectrum. Theseembodiments may make the detection less obtrusive.

In other embodiments other known techniques for signal extraction of theheart pulse may be utilized, for example, techniques which rely on smallmotions of the face (cardio-ballistography).

In some embodiments techniques such as Principal Component Analysis(PCA) and Independent Component Analysis (ICA) may be used. Othertechniques that may be used include decomposition of the pulse componentfrom the detected signals.

Both the (partial) motion of the human face as well as variations in theenvironmental light will distort the spatial and temporal homogeneitymeasures. In some embodiments the detector may measure and quantify bothmotion and light variation of the subject. In some embodiments, thedetection decision in step S206 may be disabled if motion and/or lightvariation exceeds predefined values. Alternatively, the threshold levelsfor the homogeneity measures may be altered. For example a detectiondecision may be more relaxed in case of increasing amounts of motionand/or light variation.

In some embodiments instead of basing homogeneity measure H3 oncorrelation matrix calculated from the waveforms, the following may beconsidered for checking if the waveforms of regions are time aligned.For each signal the phase of the dominant frequency may be calculated.For a real face the phase values may only differ marginally over theface area. In some embodiments, the signals for different block sizes(scales) may be calculated. The dominant frequency for a block for acertain scale may be close to the dominant frequency of an overlappinglarger block (at a higher scale), this may indicate that the signals ofneighbouring blocks at a single scale are both time and frequencyaligned.

In some embodiments, instead of comparing the homogeneity feature valuesH1 to H5 with a predefined threshold value individually, a classifiermay be used to take the decision if a real face is present. Theclassifier may take the set of feature values as input, and take theclassification decision based on determining the probability that theset of feature values may occur for a real face. By observing thefeature values in combination, the classification decision may be mademore accurately than observing each feature value individually.

In some embodiments two or more homogeneity metrics may be combinedspatially and/or temporally.

In some embodiments the homogeneity value for time correlation H3 may bebetween each pair of signals. In some embodiments only sufficientlyreliable homogeneity values may be used. This homogeneity value may befrom a correlation of phases between the signals rather than a constantphase. A constant phase may be easier to spoof and in practice the HRvaries over time.

In some embodiments the homogeneity value H4 may be for a totalmeasurement period and may be subdivided into overlapping timeintervals. The HR may be calculated for each interval. The variation ofthe HR may then be analysed over all time. If the face being measured isfrom a mask, the method used to generate H4 may give random values witha large spread. If the face being measured is real, the method used togenerate H4 may give a far lower spread.

In some embodiments the homogeneity value H5 may be for direction incolour space, for example this may be 3D colour space (e.g.RGB—red/green/blue). In some embodiments the variation for the HR may beinvestigated in for example the red axis over time. Compare to expectedvariation from absorption of HbO₂ i.e. colour errors. From thehaemoglobin absorption spectra one can expect to extract the“heart-beat” vector moving along specific direction in the RGB colourspace. If variations are detected but do not correspond to the expecteddirections in colour space, the likelihood of spoofing is increased.

In other embodiments, the homogeneity value H5 may not be limited to 3Dcolour space, for example using a hyperspectral camera. For a real facethe variation in reflected light spectrum may match the absorptionspectrum of haemoglobin (after correction for spectrum of light sourceand for skin pigmentation), if the difference between both spectra istoo large spoofing may be assumed.

Homogeneity values H1-H5 may be more appropriate when detectingdifferent spoofing methods. For example values H1-H4 may be particularlyuseful to detect a mask, whereas H5 may be particularly useful to detecta varying light signal.

As the homogeneity values H1, and H2 are correlated, in some embodimentsthe system can chose between using either.

In some embodiments all the aforementioned techniques may be utilized.In others, two or more of the above mentioned techniques may be used.

It should be appreciated that the one or more techniques selected may bedependent on the application.

In some embodiments a face detection algorithm may be taught or trainedto detect a real face by showing the system one or more real faces, andone or more fake faces.

In some embodiments the face detection algorithm may be taught ortrained to weight the homogeneity values against a neural networkdecision or a support vector machine.

In some embodiments the face detection algorithm may utilize acomparison of a currently measured set of homogeneity values againstpreviously measured homogeneity values.

In some embodiments the face detection algorithm may utilize aclassification system of the homogeneity values.

In some embodiments any body areas/parts may be used to detect vitalsigns.

In some embodiments the vital sign used to determine whether a face isreal may be a person's breathing motion.

Reference is made to FIG. 5 which shows on the left an image of twofaces. The image referenced 50 a is a real face and the image referenced52 a is that of a dummy made of plastic. The image on the right shows arespective HR map 50 b and 52 b for the two images.

The standard deviation and maximum may be taken from the face as a wholein homogeneity values H1 and H2. If the face is real, the histogramdistribution will be narrow with a strong peak. If the face is a mask ora photo, frequencies would be randomly and evenly distributed over thehistogram.

Correlation between waveform signals from different areas of the facemay detect partial masks. This may additionally or alternatively helpwith illumination issues.

The colour vector method disclosed earlier allows the detection ofspoofer sending a signal to simulate a heart-beat.

Consider the following analysis of this data using standard deviation.

From the heart-rate map of FIG. 5, the standard deviation is calculatedfor each rectangle and the homogeneity measure (according to Eq. 1) iscalculated, leading to:H_face_left=10 bpm;H_face_right=45 bpm;

Setting the threshold value (T) to 20 ensures that the face on theright, the dummy face, is rejected.

Consider the following analysis of this data using energy contained inthe histogram. Reference is made to FIG. 6 which shows the imagereferenced 50 b and the corresponding histogram 312 of heart rateagainst energy as well as the image reference 52 b and the correspondinghistogram 311. From the heart-rate map of FIG. 5 or 6, the histogram ofheart-rate values may be computed for each rectangle and the homogeneitymeasure (according to Eq.2) may be calculated:H_face_left=0.6;H_face_right=0.084;

The histogram's peak originating from the left rectangle (real face)contains more energy compared to the histogram's peak originating fromthe right rectangle (dummy face). It should be noted that the energyaxis of the two histograms have different scales.

Setting the threshold value (T) to 0.5 ensures that the face on theright, the dummy face, is rejected.

Consider the following analysis of this data using correlation oftemporal signals. The detected waveforms from the left rectangle (realface) are correlated (one with all others); the same process may beapplied for the right rectangle (dummy face). The homogeneity measure(according to Eq.3) may be extracted for each rectangle:H_face_left=0.79;H_face_right=0.39;

Setting the threshold value (T) to 0.5 ensures that the face on theright, the dummy face, is rejected.

Reference is now made to FIG. 7 which shows an embodiment of the presentmethod for verifying whether a face is real.

In step S51, it is determined for a plurality of spatially separatedregions of a surface, a respective measure of at least one vital sign.

In step S52, it is determined from the respective measures of said atleast one vital sign, homogeneity information associated with therespective measures

In step S53, the homogeneity information is used to determine if saidspatially separate regions of said surface are living tissue.

Embodiments have many applications. For example, some embodiments mayhave application in the high security sector. Other embodiments may haveapplication in consumer computer applications where a biometrics inputis required.

Some embodiments may be used where identification of a person is basedon face recognition using a camera. For example, some embodiments mayprovide authentication for logging into a smartphone using facerecognition, or verification of a person's identification at an airportterminal through computerised means.

Some embodiments may be used for non-facial biometric recognitiontechniques, for example, finger print, or palm print recognition.

Aspects of the embodiments may be implemented in a computer programproduct, which may be a collection of computer program instructionsstored on a computer readable storage device which may be executed by acomputer. The instructions may be in any interpretable or executablecode mechanism, including but not limited to scripts, interpretableprograms, dynamic link libraries (DLLs) or Java classes. Theinstructions can be provided as complete executable programs, partialexecutable programs, as modifications to existing programs (e.g.updates) or extensions for existing programs (e.g. plugins). Moreover,parts of the processing of the present invention may be distributed overmultiple computers or processors.

Storage media suitable for storing computer program instructions includeall forms of non-volatile memory, including but not limited to EPROM,EEPROM and flash memory devices, magnetic disks such as the internal andexternal hard disk drives, removable disks and CD-ROM disks. Thecomputer program product may be distributed on such a storage medium, ormay be offered for download through HTTP, FTP, email or through a serverconnected to a network such as the Internet.

Various embodiments with different variations have been described hereabove. It should be noted that those skilled in the art may combinevarious elements of these various embodiments and variations.

Such alterations, modifications, and improvements are intended to bepart of this disclosure, and are intended to be within the scope of thepresent invention. Accordingly, the foregoing description is by way ofexample only and is not intended to be limiting. The present inventionis limited only as defined in the following claims and the equivalentsthereto.

The invention claimed is:
 1. A computer implemented method comprising:determining respective measures of at least one vital sign for aplurality of spatially separated regions of a surface using at least oneimage; determining, from the respective measures of the at least onevital sign, homogeneity information associated with the respectivemeasures; and determining when the spatially separate regions of thesurface are living tissue based on the determined homogeneityinformation.
 2. The method as claimed in claim 1, wherein the at leastone image comprises a sequence of video frames, the method furthercomprising identifying a face area in the sequence of video frames,wherein at least one of the plurality of spatially separated regions ispositioned in the face area, and the surface comprises at leastpartially the face area.
 3. The method as claimed in claim 2, whereinthe at least one vital sign comprises a heart-beat signal, said methodfurther comprising extracting, from the sequence of video frames, aheart-beat signal for each of the spatially separated regions so as toobtain a plurality of heart-beat signals.
 4. The method as claimed inclaim 3, wherein determining the homogeneity information comprisescomparing the heart-beat signals.
 5. The method as claimed in claim 3,wherein determining the homogeneity information comprises extracting aheart-rate from the heart-beat signals for each of the plurality ofspatially separate regions combining the extracted heart-rates from theplurality of the spatially separate regions, and determining at leastone of a standard deviation and a maximum value of the combinedheart-rates.
 6. The method as claimed in claim 3, wherein determiningthe homogeneity information comprises finding a correlation between atleast one pair of the heart-beat signals.
 7. The method as claimed inclaim 3, wherein determining the homogeneity information comprisesextracting heart-rates from a plurality of time segments of a spatiallyseparate region of the plurality of spatially separate regions anddetermining a standard deviation of the heart-rates extracted from eachtime segment.
 8. The method as claimed in claim 3, further comprisingdetermining an area in the sequence of video frames where respiration ismeasurable, extracting a respiration signal from the determined area,extracting a correlation between the extracted respiration signal andthe heart-beat signal, and comparing the correlation to a limit.
 9. Themethod as claimed in claim 1, wherein determining when the spatiallyseparate regions of the surface are living tissue comprises comparingsaid homogeneity information against a limit.
 10. The method as claimedin claim 1, further comprising determining a colour vector for each of aplurality of time segments of a spatially separate region of theplurality of spatially separate regions and determining variations ofthe colour vectors over the plurality of time segments.
 11. The methodas claimed in claim 10, wherein the colour vector is obtained from ahaemoglobin absorption spectrum.
 12. A spoofing detection apparatuscomprising at least one processor; and a non-transitory memory forstoring instructions that when executed by the at least one processor,cause the at least one processor to: determine respective measures of atleast one vital sign for a plurality of spatially separated regions of asurface; determine from the respective measures of the at least onevital sign homogeneity information associated with the respectivemeasures; and determine when the spatially separate regions of thesurface are living tissue based on the homogeneity information.
 13. Thespoofing detection apparatus of claim 12, further comprising: an inputto receive a sequence of video frames, wherein the instructins furthercause the at least one processor to, identify a face area in thesequence of video frames, identify spatially separate skin portions inthe face area as the plurality of spatially separated regions of thesurface, extract a heart-beat signal for each of the spatially separateskin portions, and to extract a heart-rate from each heart-beat signal.14. The spoofing detection apparatus of claim 13, wherein theinstructions further cause the at least one processor to calculate avalid vital sign metric and compare the calculated valid vital signmetric against a limit to decide if the sequence of video framescontaining the face area has been captured directly from the face of aliving person.